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ABSTRACT 


The eight science and engineering practices outlined in the Next Generation Science Standards 
(NGSS) are key to student science learning. However, there is limited guidance for teachers on 
what these practices “look like” in instruction across different grade ranges and topic areas. 
Further, there is little existing guidance for teachers on how to make the practices in the NGSS 
accessible to all students, particularly those from groups that have traditionally been underserved 
in science education. 


This paper describes the development of a primer that operationalizes the science practices for 
different grade levels and content areas. This work is part of a larger study aimed at developing 
survey items to measure teachers’ use of the science practices. 
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INTRODUCTION 


The Next Generation Science Standards (NGSS) provide a vision of science instruction in which 
all students develop scientific literacy, and promote a blending of science and engineering 
practices with disciplinary core ideas and crosscutting concepts (NGSS Lead States, 2013). The 
three dimensions are intertwined such that students learn targeted content ideas through engaging 
in the practices to develop explanations for real-world phenomena. The eight practices outlined 
in the NGSS—asking questions/defining problems; developing and using models; planning and 
carrying out investigations; analyzing and interpreting data; using mathematics and 
computational thinking; constructing explanations/designing solutions; engaging in argument 
from evidence; and obtaining, evaluating, and communicating information—are, thus, critical to 
students understanding science and engineering. 


The NGSS draw from National Research Council (NRC) reports indicating that students from 
diverse backgrounds are able to successfully engage in the practices and understand science 
concepts when they are provided with equitable learning experiences (NRC 2007, 2008, 2011). 
However, research indicates that this vision of providing equitable access to quality science 
learning experiences is far from being realized. Data from 2012 National Survey of Science and 
Mathematics Education (NSSME) show that well-prepared teacher are distributed unevenly 
across schools with different student populations, with disparities both in the amount of 
experience teachers have and in their professional preparation. For example, classes with high 
percentages of students historically underrepresented in STEM and classes in high-poverty 
schools are more likely to be taught by novice teachers than their counterparts. Additionally, 
science teachers in high-poverty schools are less likely than those in low-poverty schools to have 
a science/science education degree (Smith, Trygstad, & Banilower, 2016). Given these types of 
unequal access to education resources, the persistent achievement gaps found in the National 
Assessment of Educational Progress are not surprising (National Center for Education Statistics, 
2012). 


Achieving the goals of the NGSS will require long-term effort and significant changes to how 
new teachers are prepared, professional learning opportunities for in-service teachers, 
instructional materials, and assessments, as well as other aspects of the science education 
infrastructure (National Science Teachers Association, 2013). In addition, and foundational to 
these efforts, guidance will be needed on what the practices “look like” when they are 
operationalized in instruction. As the implementation of the NGSS unfolds, it will be essential to 
measure NGSS implementation, and examine the extent to which students are being provided 
with equitable learning experiences. 
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This paper describes one component of a larger study aimed at developing survey items to 
measure student opportunity to engage in the science practices, with the ultimate goal of 
monitoring progress towards widespread student engagement with the science practices. 
Specifically, the development of a primer is describe that characterizes key elements of the 
science practices’ and provides illustrative examples of how these key elements play out in 
instruction across different grade bands (K—2, 3-5, 6-8, 9-12) and subject areas (biological 
sciences, Earth sciences, physical sciences). We envision the primer providing fodder for the 
needed national discussion about implementing the NGSS that will require developing a 
common vocabulary to discuss the practices. The key elements of the practices also serve as the 
basis for the survey development, which ultimately will be one of several tools for monitoring 
student opportunity to experience high-quality science instruction. 


PROCEDURE 


Two methods of data collection were used to inform development of the primer. First, an 
extensive review of existing literature was conducted on how the practices can be incorporated 
into classroom instruction. Second, an online, modified-Delphi panel (Miller & Pasley, 2012) 
was convened to identify key aspects of instruction that students need to experience for each of 
the science practices at different grade bands and areas of science. These activities are described 
further in the following sections. 


Literature Review on the Practices 


A literature review was conducted to identify and summarize current research and practice-based 
knowledge focused on engaging students with the science and engineering practices. A search 
utilizing the ERIC and Google scholar databases was conducted to find articles on the NGSS 
practices published since the release of the Framework for K-12 Science Education in 
2011(NRC, 2011) as we assumed that relevant work that preceded it was already incorporated in 
the Framework. 


Key words and phrases (e.g., science and engineering practices, NGSS, models, argumentation, 
explanations) were used in conjunction, and the initial search resulted in 76 empirical and 
practitioner articles and conference papers. These articles were reviewed more carefully and 
articles that included only general descriptions of the practices were removed from the literature 
pool. The remaining 47 articles were coded by the practices, grade levels, and subjects 
addressed, when possible. 


' Presently, project efforts are focused only on the science practices. However, it is possible that this work may 
expand to the engineering practices in the future. 
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References to all of the articles included in this literature review, coded to the practices they 
address, can be found in the Appendix. The majority of the articles that passed the screening 
were from practitioner journals (35 articles). Articles intended for high school teachers (16) 
were the most prevalent, followed by middle school (12), and elementary (7). Frequently these 
practitioner articles focused on one or two of the practices, detailing how they play out in the 
classroom. The remaining 12 articles were either empirical or products of conference 
presentations. The description of the practices culled from these articles was used as a 
foundation for the questions posed in the subsequent panel. 


Expert Panel 


An expert panel was convened to “unpack” the science practices, identifying key elements of the 
practices in different grade bands and areas of science. The panel was composed of individuals 
with backgrounds and experiences in different grade levels and content areas. In addition, 
panelists had varied levels of involvement with the NGSS, including some panelists who served 
on the writing team. 


The panel process involved four rounds of questions that were emailed to the panelists. 
Questions in Rounds One and Two addressed four practices: constructing explanations; engaging 
in argument from evidence; obtaining, evaluating, and communicating information; and using 
mathematics and computational thinking. Rounds Three and Four focused on the remaining 
practices: developing and using models, asking questions, planning and carrying out 
investigations, and analyzing and interpreting data. Panelists were asked to complete a series of 
questions/tasks related to each practice including: (1) constructing a definition of the practice; (2) 
identifying the key elements of what students need to experience at various grade levels and in 
different science content areas that are essential for them to understand the practice; and (3) 
generating examples of what instruction “looks like” as students engage with the practice. 


Constructing a Definition of the Practice 
Panelists were provided with a draft definition of each science practice drawn from the 


Framework, NGSS, and various articles collected as part of the literature review. They were 
asked to review the draft, suggest revisions to the definition they deemed as necessary, and 
provide a rationale for any revisions they recommended (see Figure 1). The definitions provide 
by each panelist was used as a lens to interpret their remaining responses to questions/tasks 
related to that practice. 
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The Next Generation Science Standards (NGSS) and various articles characterize the practice 
of asking questions. From those sources, we have developed the following definition: 


Asking questions is the science practice of generating queries about phenomena, systems 
(or parts of systems), or proposed explanations or models that can potentially be 
answered with explanations supported by empirical evidence. 


a. Use track changes to make any revisions to the definition that you think are 
necessary. 


Asking questions is the science practice of generating queries about 
phenomena, systems (or parts of systems), or proposed explanations or models 


that can potentially be answered with explanations supported by empirical 
evidence. 


b. Please provide a rationale for the revisions that you made to the definition. 


Figure 1 


Identifying the Key Elements of the Practice 
For each science practice, panel members were asked to provide a detailed example of 


instruction engaging students in the practice as part of meeting an NGSS performance 
expectation or learning about a particular topic area. Panelists were then asked to identify the 
key elements of what students were doing in their description that are essential for helping 
students master that practice. In addition, panelists were asked to specify whether/how the key 
elements differed by grade band and science content area. The key elements offered by the 
panel, along with other key elements identified in the literature review, were compiled. 


In the subsequent round, panelists were asked to provide feedback on the set of compiled key 
elements, noting whether they: (a) agreed that the element is essential for helping students master 
the practice; (b) agreed that the element is essential for helping students master the practice if 
modified (providing the edits needed); or (c) disagreed that the element is essential for helping 
students master the practice (see Figure 2). Panelists also had the opportunity to add key 
elements that were missing from the compiled list and to specify at which grade bands students 
should experience each of the listed elements. 
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Feedback on Key Elements of Planning and Carrying Out Investigations 


For each element listed, we would like you to do three things. First, indicate whether you (a) 
agree that it is essential for helping students master the practice, (b) agree that it is essential for 
helping students master the practice if modified: or (c) disagree that it is essential for helping 
students master the practice. (If you select option b, please indicate the edits needed along with a 
rationale for the changes. If option c is selected, please provide a rationale.) Second. indicate at 
which grade bands students should experience the element. Third, describe how the level of 
complexity with which students should experience the element varies at different grade bands. 


Key Element: Identify what evidence is required to answer a scientific question 
1. Which best describes your agreement with the key element listed? (Please select one of 
the three options below by either checking the box to the left or highlighting the 
response.) 


a. O Lagree that this element is essential for helping students master the practice of 
planning and carrying out investigations. 


b. O Lagree that this element is essential for helping students master the practice of 
planning and carrying out investigations if modified. 


Please provide the edits you would suggest along with a rationale for the changes. 


c. OC) Idisagree that this element is essential for helping students master the practice of 
planning and carrying out investigations. 


Please provide an explanation for why you disagree. 


bo 


At which grade band(s) should students experience this element, regardless of the 
complexity with which they experience it? (Please select all that apply by either checking 
the boxes to the left or highlighting the responses.) 

CO K-2 

OC 3-5 

0 6-8 

OC 9-12 


3. Describe how the complexity of this element varies at different grade bands. (If you do 


not believe the — varies across = bands, — = “NA. ”’) 


Figure 2 
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Generating Examples of Students Engaging with the Science Practices 
Throughout the four rounds, panelists were asked to generate descriptive examples of instruction 


in order to operationalize what instruction would “look like” when student experience a 
particular practice (see Figure 3). As noted previously, panelists provided descriptive examples 
in order to generate a list of key elements. In addition, panelists were asked to use the final list 
of key elements to provide a detailed example of students engaging with a science practice in 
order to illustrate what the practice looks like at a particular grade band and identify differences 
across grade bands. 


Example of Analyzing and Interpreting Data 


Considering the revised list of elements for analyzing and interpreting data that you have 
specified in this round (i.e., all of the elements you have agreed with, edited, or added), please 
provide the following at a grade band and content area of your choice: 


a. A detailed example of students analyzing and interpreting data that illustrates 
how students experience this practice at the selected grade band. Please be 
specific in your illustration. 


b. A description of how your example illustrates the key elements of analyzing and 
interpreting data. 


On what grade band does your description focus? (Select a grade band by either checking the 
box to the left or highlighting the response.) 


LC) K-2 
OD 3-5 
LC) 6-8 
C) 9-12 


Detailed example for analyzing and interpreting data 


Description of how example illustrates the key elements of analyzing and interpreting data 


Figure 3 
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DATA ANALYSIS 


Data gathered during the expert panel process were analyzed in an ongoing fashion during the 
data collection period. The first round of panel responses were analyzed in order to frame the 
second round of questions, an iterative process that continued through all rounds of the panel. 
Use of the three strategies (constructing a definition of the practice, identifying key elements of 
the practice, and generating examples of students learning science through engaging with the 
science practices) led to a purposeful accumulation of data and vetting of emerging themes. The 
goal in data analysis was to identify and verify key elements of a practice, meaning that they 
were ideas that panelists largely agreed were important for the practice under discussion. 


In each round of the panel, responses to each question posed produced a data set that was 
analyzed. Project staff constructed a simple thematic coding scheme for analyzing each item in 
each round and tracking the strength of agreement (e.g., similar responses for an item). In 
addition to the reflections that panelists offered in each round of the panel, the examples that 
they provided were treated as data to be coded. These examples were analyzed as short 
narratives of particular practices in terms of the relationships illustrated or explicated. 


FINDINGS 


The process of reviewing the literature review on the practices, in conjunction with the expert 
panel process, yielded set of key elements for each science practice that operationalize what 
student should experience as part of engaging in a particular practices.” In the process of 
working with the panel and reviewing the literature, a number of findings emerged which have 
important implications for widespread implementation of the NGSS and developing instruments 
to measure the extent of implementation. 


There is variation in how teachers, teacher educators, researchers, and other 
stakeholders view the nature and role of the practices. 
The panel process revealed substantial differences in how people define the science practices and 


describe “what it looks like” for students to engage with a particular practice. For example, 
panelists varied on what constitutes a “model,” with some agreeing that models are physical, 
graphical, or mathematical representations of phenomena while another noted that models can 
also be mental representations. There were also differences in how panelists viewed the purpose 
of particular practices. For example, some panelists interpreted the practice of “asking 
questions” as those questions that lead to an investigation. Other panelists had a broader view of 
questions, recognizing the practice of questioning in developing an investigation, but also 
recognizing the role of questioning in other science practices such as clarifying evidence or the 


The primer will include definitions and key elements for all of the science practices. The primer is in the final 
stages of development, and will be available on our website soon: www.horizon-research.com. 
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premise of an argument. In addition, there was an inherent tension between describing the 
practices alone for clarity and incorporating their descriptions with disciplinary core idea and 
crosscutting concepts. 


Relatedly, it quickly became apparent that even among the expert panel, key terms related to the 
practices are used in different ways. For example, panelist varied in how they used the terms 
“claims,” “arguments,” and “explanations.” As a result, it was difficult to obtain total consensus 
on what it “looks like” to engage with the science practices in the context of a K-12 instruction, 
particularly in the computational thinking. 


Lack of clarity about what constitutes engaging in the practices will present a challenge to 
achieving high-quality implementation of the NGSS throughout the nation as teachers will likely 
need extensive guidance on how students should experience the NGSS practices in K-12 
instruction. This issue was also apparent in cognitive interviews conducted with teachers as part 
of developing items to measure implementation of the practices. For example, “computational 
thinking” is a particularly problematic phrase. Interviewed elementary teachers often defined 
computational thinking as doing computation—addition, subtraction, multiplication, and 
division. At the secondary level, teachers described computational thinking as anything 
involving computers, including simply using a word processor. The experience of this project 
points to the need to provide clear guidance for teachers in implementing the practices in their 
classrooms, and resources that clearly describe how students should engage with the practices 
and clarify terms used in these descriptions. For this reason, the primer includes definitions of 
the practices with an unpacking of what students should experience as part of engaging with each 
practice (the key elements). In addition, it includes a glossary of important terms used in these 
descriptions. Our hope is that these definitions and descriptions will foster productive 
conversations around the practices and create a broader, consensus understanding of their role in 
science education. 


The practices often have overlapping elements and are used concurrently in 
science. 
Because the practices overlap as they are used in science, it was sometimes difficult for panelists 


to tease apart key elements for individual practices. For example, elements of using mathematics 
and computational thinking often occur as part of analyzing and interpreting data. Although 
deconstructing the practices into key elements is necessary for understanding what is involved in 
engaging in a practice, we recognize that it is inauthentic to consider key elements and practices 
in isolation, as the practices are often used in connection with each other. For this reason, the 
primer includes vignettes that illustrate how instruction that incorporates the key elements might 
appear in different subject areas and grade bands. 
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The ultimate goal of science is the development of evidence-based explanations 
for, and models of, the natural world. The science practices are in the service of 
these goals, and critical to students gaining an understanding of how scientific 
knowledge is generated. 

Although the practices are intimately intertwined, the unpacking process highlighted the danger 


that the ultimate goal of science could get lost by focusing on individual practices. The science 
practices are critical to achieving this goal, as students need to understand that scientific 
evidence is generated through a systematic and social process. Thus, it is important that teachers 
keep this ultimate goal of science in mind when planning and delivering instruction so that 
students develop an understanding of how the practices are in service of this goal. 


The science practices apply to all fields of science and grade levels. 
Throughout the rounds of the expert panel, panelists consistently agreed that (a) the practices do 


not differ across topics/subject areas, and (b) student engagement in the practices grows 
progressively more complex as grade level increases. However, as the key elements of the 
practices were further unpacked, it became apparent that some key elements are too sophisticated 
for students in the lowest grade bands to engage with in authentic ways, particularly some 
aspects of mathematics and computational thinking. It also became evident that as students 
progress through the grade bands, it may become unnecessary for teachers to explicitly engage 
student with some of the key elements of practices since students will have sufficiently engaged 
with particular aspects of a practice in earlier grades. For example, assuming students have 
learned to distinguish scientific and non-scientific questions in previous years, high school 
teachers will likely not have to emphasize this aspect of asking questions. 


Students need to have opportunities to reflect on their use of the practices. 
In addition to having opportunities to engage in the science practices, it became apparent that 


students should also have periodic opportunities to engage in metacognition about their learning 
and be asked to explicitly reflect on their use of various practices. For example, if students use 
different types of models (e.g., descriptive, relational) for various purposes (e.g., to make 
predictions, to compare with other models) over the course of their 8" grade science class, the 
teacher may want to help students reflect on the various types models they have used and discuss 
the purpose of models more broadly in science. 


DISCUSSION 


The primer, developed through an extensive literature review and expert panel process, provides 
a much needed opportunity to further the discussion about how NGSS-aligned instruction, in 
particular the science practices, can be implemented in K-12 classrooms. The primer sets forth 
definitions of each practice, key elements of what student engagement in each practice would 
involve, and illustrative examples of how these key elements play out during instruction. The 
lessons learned in the process of creating the primer highlight the need for, and value in, the in- 
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depth discussions, informed by both the literature and practitioner knowledge, about 
implementing the NGSS in classrooms. They also highlight the importance of providing 
concrete and accessible examples NGSS-aligned instruction as objects of inquiry for these 
discussions. Given the historic inequities in science education, making these types of resources 
widely available may be an important first step in closing the equity gap. 


The development of the primer also had a number of implications for development of the survey 
that is the ultimate goal of this study. That the panel agreed that the key elements of each 
practice applied to all content areas indicated that it was not necessary to develop different items 
for different science topics. However, the level of complexity at which students might engage 
with a practice would be expected to vary across grades bands, indicating that separate items 
might be needed for teachers at different grade levels. 


A second challenge highlighted by the primer development was the wide variation in how people 
view the nature and role of the practices. For survey items to be valid, respondents must 
interpret the questions as intended. The variation in thinking about the practices that we 
encountered in the literature and panel process foreshadowed a number of issues we uncovered 
in cognitive interviews with teachers around draft survey items. 


A third implication is that survey items related to metacognition around the practices would be a 
valuable tool for monitoring implementation of the NGSS. The panel agreed on the importance 
of teachers providing opportunities for students to not only engage in the practices to develop an 
understanding of important science ideas, but to also reflect on how the practices were enabling 
them to develop those understandings so that they would both better understand how scientific 
knowledge is generated and revised and gain facility with the diverse methods of scientific 
inquiry. Although we are not able to tackle this aspect of instruction in our current study, it 
clearly points to an area in need of further work. 
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Carrying Out 
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Interpreting 
Data 


Using 
Mathematics & 
Computational 

Thinking 


Constructing 
Explanations 


Engaging in 
Argument 
from 
Evidence 


Obtaining, 
Evaluating, & 
Communicating 
Information 
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2012 


Xx 
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Brodsky & Falk, 2014 
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Oh, 2013 
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2013 


Chowning & Griswold, 
2014 


Clary & Wandersee, 
2013 
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Anderson, 2013 


Cunningham & Carlsen, 
2014 
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Dash & Hug, 2014 


Falk & Brodsky, 2013a 


Falk & Brodsky, 2013b 


Falk & Brodsky, 2014 


Ford, 2012 
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Foster, Husman, & 
Mendoza, 2013 


Gotwals & Songer, 
2013 


Gould, Sunbury, & 


Dussault, 2014 


Huff & Bybee, 2013 
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Asking & Using Carrying Out | Interpreting | Computational | Constructing from Communicating 
Questions Models Investigations Data Thinking Explanations | Evidence Information 
Knight & Grymonpre, x 
2013 
Krajcik & Merritt, 2012 Xx 
Kraus, 2014 Xx Xx 
Krell, zu Belzen, & x 
Kriiger, 2014 
Lachapelle, Sargianis, 
& Cunningham, Xx Xx Xx Xx xX Xx Xx Xx 
2013 
Lee, Cite, & Hanuscin, x 
2014 
Lee, Quinn, & Valdés, 
013 Xx Xx Xx Xx 
Llewellyn, 2013 Xx 
Llewellyn & Rajesh, x 
2011 
Mayer, Damelin, & x 
Krajcik, 2013 
Mayes & Koballa, 2012 Xx Xx X Xx Xx Xx Xx 
McLaughlin, 2013 Xx 
McNeill, 2011 Xx Xx 
Miele & Bennett, 2014 x x x 
Milano, 2013 Xx Xx Xx Xx Xx Xx Xx Xx 
Miller, Rivet, Kastens, x 
& Lyons, 2013 
Miranda & Hermann, 
2013 Xx Xx Xx Xx Xx Xx 
Oh & Oh, 2013 Xx 
Osborne & Patterson, 
2011 * Se 
Reiser, Berland, & 
Kenyon, 2012 a S 
Rivet & Kastens, 2012 xX 
Sampson, Enderle, & x 
Grooms, 2013 
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